PT-PVM: Using PVM in a multi-threaded environment

This paper describes an extension to the PVM message passing library. PTPVM provides message passing and process management facilities at thread level for a cluster of workstations running the UNIX operating system without changing the PVM system. In the first place these extensions are used to implement the Coordination Language COLA which is used to model the behavior of processes in a massively parallel environment. The system serves as the base communication and lightweight process management platform to validate the feasibility of the abstractions introduced in COLA. 1 Motivation The coordination language COLA [CG92, BHK94] focuses on the coordination of highly asynchronous concurrent processes running in a massively parallel environment. The COLA computation space consists of a large collection of fine grained processes communicating using the asynchronous message passing paradigm. COLA processes are dynamic which means that during runtime of an application new processes can be created or destroyed randomly. We do not make any assumption on the runtime behavior of COLA processes, once started they can compute forever or can immediately block on a coordination feature. This implies that the underlying process model of the software platform where COLA is built on should support a preemptive fine grained lightweight process scheduling management. Because the standard distribution of PVM (version 3.3.7) is not thread safe, (not reentrant) and because its computing model is restricted to heavyweight UNIX processes, we developed PT-PVM1 which will be described in the following sections. The reminder of this paper is organized as follows: Section 2 explains the system model the platform is built on, Section 3 gives an overview of the performance results obtained using this platform, Section 4 finally concludes the paper and gives an outlook on our plans for the future. 2 System Model This section describes the basic programming abstractions which are used to build PT-PVM followed by an overview of its implementation (see Figure 1). 2.1 Programming Abstractions The programming model of PT-PVM is based on three abstractions which follow directly from the implementation of different thread packages available for UNIX platforms. These abstractions determine the location of a running PT-PVM thread on a cluster of workstations: the parallel virtual machine, controlled by PVM [Sun90], consisting of several CPUs; one or more process environments (PE) running on severalCPUs; aPE serves as the “heavy-host” process for a PT-PVM thread (similar to the ”pods” of [FS95]); one or more PT-PVM threads running on differentPEs denoted by aTID (thread identifier). Therefore, a PT-PVM thread will be specified using a triple: (CPU; PE; T ID). The user of PT-PVM has to specify at initialization phase of the environment how many CPUs should be used and how many PEs should be invoked on these CPUs, the distribution of the threads onto the (CPU; PE) space will be done transparently by the system. In order to extend the computing model of PVM onto thread level we borrow the concept of Correspondents from COLA [OKR94]. A Correspondent is a high level identification concept for purpose of communication of processes; in this concrete implementation it consist of a tuple: (PE; TID). We distinguish two types of Correspondents depending on the invocation strategy of the new thread: local Correspondent for local invocation and non-local Correspondent for distributed invocation (see also Section 2.2). To use non-reentrantcode (PVM source) in a multi-threaded environmentwith preemptive threads, one has to make sure that on each PE only one instance (thread) is using PVM primitives at a time. This is achieved using a client server approach: on each 1PT-PVM stands for Preemptive Threads and PVM. PE PE PE PE PE PE PT-PVM User Thread Message Router handling messages for this PE